Dynamic Emotion Modeling With Learnable Graphs and Graph Inception Network

Human emotion is expressed, perceived and captured using a variety of dynamic data modalities, such as speech (verbal), videos (facial expressions) motion sensors (body gestures). We propose generalized approach to recognition that can adapt across modalities by modeling structured graphs. The motivation behind the graph build compact models without compromising on performance. To alleviate problem optimal construction, we cast this joint learning classification task. end, present Learnable Graph Inception Network (L-GrIN) jointly learns recognize identify underlying structure in data. Our architecture comprises multiple novel components: new convolution operation, inception layer, learnable adjacency, pooling function yields graph-level embedding. evaluate proposed five benchmark databases spanning three different (video, audio, capture), where each database captures one following emotional cues: facial expressions, body gestures. achieve state-of-the-art performance all outperforming several competitive baselines relevant existing methods. shows superior with significantly fewer parameters (compared convolutional or recurrent neural networks) promising its applicability resource-constrained devices. code available at https://github.com/AmirSh15/graph_emotion_recognition.